Specify charset when using MySQL #1059

greg0ire · 2022-02-16T09:22:28Z

Q	A
License	MIT
Doc PR	symfony/symfony-docs#16515

Because of backward-compatibility reasons, not specifying the charset
will result in "utf8" being used, which is a deprecated charset.
Also, when the charset for creating tables is not specified, the one
specified in the DSN is reused if specified.

In the future, DBAL is going to drop default values, which means we will
have to specify a charset anyway.

See:

Recommend a better charset by default doctrine/dbal#5277 (about the reuse of connection charset for table charset)
Remove defaults for MySQL table charset, collation and engine doctrine/dbal#4644

github-actions · 2022-02-16T09:22:54Z

Thanks for the PR 😍

How to test these changes in your application

Define the SYMFONY_ENDPOINT environment variable:

# On Unix-like (BSD, Linux and macOS)
export SYMFONY_ENDPOINT=https://raw.githubusercontent.com/symfony/recipes/flex/pull-1059/index.json
# On Windows
SET SYMFONY_ENDPOINT=https://raw.githubusercontent.com/symfony/recipes/flex/pull-1059/index.json

Install the package(s) related to this recipe:

composer req 'symfony/flex:^1.16'
composer req 'doctrine/doctrine-bundle:^2.4'

Don't forget to unset the SYMFONY_ENDPOINT environment variable when done:

# On Unix-like (BSD, Linux and macOS)
unset SYMFONY_ENDPOINT
# On Windows
SET SYMFONY_ENDPOINT=

Diff between recipe versions

In order to help with the review stage, I'm in charge of computing the diff between the various versions of patched recipes.
I'm going keep this comment up to date with any updates of the attached patch.

doctrine/doctrine-bundle

1.6 vs 1.12

diff --git a/doctrine/doctrine-bundle/1.6/config/packages/doctrine.yaml b/doctrine/doctrine-bundle/1.12/config/packages/doctrine.yaml
index bea3648..9faf8e2 100644
--- a/doctrine/doctrine-bundle/1.6/config/packages/doctrine.yaml
+++ b/doctrine/doctrine-bundle/1.12/config/packages/doctrine.yaml
@@ -10,9 +10,12 @@ doctrine:
         charset: utf8mb4
         default_table_options:
             collate: utf8mb4_unicode_ci
+
+        # backtrace queries in profiler (increases memory usage per request)
+        #profiling_collect_backtrace: '%kernel.debug%'
     orm:
         auto_generate_proxy_classes: true
-        naming_strategy: doctrine.orm.naming_strategy.underscore
+        naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
         auto_mapping: true
         mappings:
             App:
diff --git a/doctrine/doctrine-bundle/1.6/config/packages/prod/doctrine.yaml b/doctrine/doctrine-bundle/1.12/config/packages/prod/doctrine.yaml
index 0a7c53b..084f59a 100644
--- a/doctrine/doctrine-bundle/1.6/config/packages/prod/doctrine.yaml
+++ b/doctrine/doctrine-bundle/1.12/config/packages/prod/doctrine.yaml
@@ -2,26 +2,14 @@ doctrine:
     orm:
         auto_generate_proxy_classes: false
         metadata_cache_driver:
-            type: service
-            id: doctrine.system_cache_provider
+            type: pool
+            pool: doctrine.system_cache_pool
         query_cache_driver:
-            type: service
-            id: doctrine.system_cache_provider
+            type: pool
+            pool: doctrine.system_cache_pool
         result_cache_driver:
-            type: service
-            id: doctrine.result_cache_provider
-
-services:
-    doctrine.result_cache_provider:
-        class: Symfony\Component\Cache\DoctrineProvider
-        public: false
-        arguments:
-            - '@doctrine.result_cache_pool'
-    doctrine.system_cache_provider:
-        class: Symfony\Component\Cache\DoctrineProvider
-        public: false
-        arguments:
-            - '@doctrine.system_cache_pool'
+            type: pool
+            pool: doctrine.result_cache_pool
 
 framework:
     cache:

1.12 vs 2.0

diff --git a/doctrine/doctrine-bundle/1.12/config/packages/doctrine.yaml b/doctrine/doctrine-bundle/2.0/config/packages/doctrine.yaml
index 9faf8e2..c319176 100644
--- a/doctrine/doctrine-bundle/1.12/config/packages/doctrine.yaml
+++ b/doctrine/doctrine-bundle/2.0/config/packages/doctrine.yaml
@@ -5,14 +5,6 @@ doctrine:
         # IMPORTANT: You MUST configure your server version,
         # either here or in the DATABASE_URL env var (see .env file)
         #server_version: '13'
-
-        # only needed for MySQL
-        charset: utf8mb4
-        default_table_options:
-            collate: utf8mb4_unicode_ci
-
-        # backtrace queries in profiler (increases memory usage per request)
-        #profiling_collect_backtrace: '%kernel.debug%'
     orm:
         auto_generate_proxy_classes: true
         naming_strategy: doctrine.orm.naming_strategy.underscore_number_aware
diff --git a/doctrine/doctrine-bundle/1.12/post-install.txt b/doctrine/doctrine-bundle/2.0/post-install.txt
index 99a8825..0e3d418 100644
--- a/doctrine/doctrine-bundle/1.12/post-install.txt
+++ b/doctrine/doctrine-bundle/2.0/post-install.txt
@@ -1,4 +1,4 @@
   * Modify your DATABASE_URL config in .env
 
-  * Configure the driver (mysql) and
-    server_version (5.7) in config/packages/doctrine.yaml
+  * Configure the driver (postgresql) and
+    server_version (13) in config/packages/doctrine.yaml

2.0 vs 2.3

diff --git a/doctrine/doctrine-bundle/2.0/config/packages/prod/doctrine.yaml b/doctrine/doctrine-bundle/2.3/config/packages/prod/doctrine.yaml
index 084f59a..17299e2 100644
--- a/doctrine/doctrine-bundle/2.0/config/packages/prod/doctrine.yaml
+++ b/doctrine/doctrine-bundle/2.3/config/packages/prod/doctrine.yaml
@@ -1,9 +1,6 @@
 doctrine:
     orm:
         auto_generate_proxy_classes: false
-        metadata_cache_driver:
-            type: pool
-            pool: doctrine.system_cache_pool
         query_cache_driver:
             type: pool
             pool: doctrine.system_cache_pool
diff --git a/doctrine/doctrine-bundle/2.3/config/packages/test/doctrine.yaml b/doctrine/doctrine-bundle/2.3/config/packages/test/doctrine.yaml
new file mode 100644
index 0000000..2ace640
--- /dev/null
+++ b/doctrine/doctrine-bundle/2.3/config/packages/test/doctrine.yaml
@@ -0,0 +1,4 @@
+doctrine:
+    dbal:
+        # "TEST_TOKEN" is typically set by ParaTest
+        dbname: 'main_test%env(default::TEST_TOKEN)%'
diff --git a/doctrine/doctrine-bundle/2.0/manifest.json b/doctrine/doctrine-bundle/2.3/manifest.json
index 062aed0..9516b21 100644
--- a/doctrine/doctrine-bundle/2.0/manifest.json
+++ b/doctrine/doctrine-bundle/2.3/manifest.json
@@ -11,13 +11,13 @@
         "#2": "IMPORTANT: You MUST configure your server version, either here or in config/packages/doctrine.yaml",
         "#3": "",
         "#4": "DATABASE_URL=\"sqlite:///%kernel.project_dir%/var/data.db\"",
-        "#5": "DATABASE_URL=\"mysql://db_user:[email protected]:3306/db_name?serverVersion=5.7\"",
+        "#5": "DATABASE_URL=\"mysql://db_user:[email protected]:3306/db_name?serverVersion=5.7&charset=utf8mb4\"",
         "DATABASE_URL": "postgresql://db_user:[email protected]:5432/db_name?serverVersion=13&charset=utf8"
     },
     "dockerfile": [
-        "RUN apk add --no-cache --virtual .pgsql-deps postgresql-dev && \\",
-        "\tdocker-php-ext-install -j$(nproc) pdo_pgsql && \\",
-        "\tapk add --no-cache --virtual .pgsql-rundeps so:libpq.so.5 && \\",
+        "RUN apk add --no-cache --virtual .pgsql-deps postgresql-dev; \\",
+        "\tdocker-php-ext-install -j$(nproc) pdo_pgsql; \\",
+        "\tapk add --no-cache --virtual .pgsql-rundeps so:libpq.so.5; \\",
         "\tapk del .pgsql-deps"
     ],
     "docker-compose": {

2.3 vs 2.4

diff --git a/doctrine/doctrine-bundle/2.3/config/packages/doctrine.yaml b/doctrine/doctrine-bundle/2.4/config/packages/doctrine.yaml
index c319176..8fe1dd4 100644
--- a/doctrine/doctrine-bundle/2.3/config/packages/doctrine.yaml
+++ b/doctrine/doctrine-bundle/2.4/config/packages/doctrine.yaml
@@ -12,7 +12,31 @@ doctrine:
         mappings:
             App:
                 is_bundle: false
-                type: annotation
                 dir: '%kernel.project_dir%/src/Entity'
                 prefix: 'App\Entity'
                 alias: App
+
+when@test:
+    doctrine:
+        dbal:
+            # "TEST_TOKEN" is typically set by ParaTest
+            dbname_suffix: '_test%env(default::TEST_TOKEN)%'
+
+when@prod:
+    doctrine:
+        orm:
+            auto_generate_proxy_classes: false
+            query_cache_driver:
+                type: pool
+                pool: doctrine.system_cache_pool
+            result_cache_driver:
+                type: pool
+                pool: doctrine.result_cache_pool
+
+    framework:
+        cache:
+            pools:
+                doctrine.result_cache_pool:
+                    adapter: cache.app
+                doctrine.system_cache_pool:
+                    adapter: cache.system
diff --git a/doctrine/doctrine-bundle/2.3/config/packages/prod/doctrine.yaml b/doctrine/doctrine-bundle/2.3/config/packages/prod/doctrine.yaml
deleted file mode 100644
index 17299e2..0000000
--- a/doctrine/doctrine-bundle/2.3/config/packages/prod/doctrine.yaml
+++ /dev/null
@@ -1,17 +0,0 @@
-doctrine:
-    orm:
-        auto_generate_proxy_classes: false
-        query_cache_driver:
-            type: pool
-            pool: doctrine.system_cache_pool
-        result_cache_driver:
-            type: pool
-            pool: doctrine.result_cache_pool
-
-framework:
-    cache:
-        pools:
-            doctrine.result_cache_pool:
-                adapter: cache.app
-            doctrine.system_cache_pool:
-                adapter: cache.system
diff --git a/doctrine/doctrine-bundle/2.3/config/packages/test/doctrine.yaml b/doctrine/doctrine-bundle/2.3/config/packages/test/doctrine.yaml
deleted file mode 100644
index 2ace640..0000000
--- a/doctrine/doctrine-bundle/2.3/config/packages/test/doctrine.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-doctrine:
-    dbal:
-        # "TEST_TOKEN" is typically set by ParaTest
-        dbname: 'main_test%env(default::TEST_TOKEN)%'
diff --git a/doctrine/doctrine-bundle/2.3/manifest.json b/doctrine/doctrine-bundle/2.4/manifest.json
index 9516b21..407a92c 100644
--- a/doctrine/doctrine-bundle/2.3/manifest.json
+++ b/doctrine/doctrine-bundle/2.4/manifest.json
@@ -12,7 +12,7 @@
         "#3": "",
         "#4": "DATABASE_URL=\"sqlite:///%kernel.project_dir%/var/data.db\"",
         "#5": "DATABASE_URL=\"mysql://db_user:[email protected]:3306/db_name?serverVersion=5.7&charset=utf8mb4\"",
-        "DATABASE_URL": "postgresql://db_user:[email protected]:5432/db_name?serverVersion=13&charset=utf8"
+        "DATABASE_URL": "postgresql://symfony:[email protected]:5432/app?serverVersion=13&charset=utf8"
     },
     "dockerfile": [
         "RUN apk add --no-cache --virtual .pgsql-deps postgresql-dev; \\",
@@ -44,5 +44,8 @@
                 "    - \"5432\""
             ]
         }
+    },
+    "conflict": {
+        "symfony/framework-bundle": "<5.3"
     }
 }

nicolas-grekas · 2022-02-16T09:52:33Z

The recipe for 2.3 should also be updated, so that projects that start on 4.4 also get the change.

Because of backward-compatibility reasons, not specifying the charset will result in "utf8" being used, which is a deprecated charset. Also, when the charset for creating tables is not specified, the one specified in the DSN is reused if specified. In the future, DBAL is going to drop default values, which means we will have to specify a charset anyway.

github-actions bot enabled auto-merge (squash) February 16, 2022 09:22

greg0ire closed this Feb 16, 2022

auto-merge was automatically disabled February 16, 2022 09:28
Pull request was closed

greg0ire reopened this Feb 16, 2022

github-actions bot enabled auto-merge (squash) February 16, 2022 09:29

derrabus approved these changes Feb 16, 2022

View reviewed changes

auto-merge was automatically disabled February 16, 2022 10:01
Head branch was pushed to by a user without write access

greg0ire force-pushed the recommend-better-mysql-charset branch from ab6057c to d3e3435 Compare February 16, 2022 10:01

github-actions bot enabled auto-merge (squash) February 16, 2022 10:01

nicolas-grekas disabled auto-merge February 16, 2022 10:04

nicolas-grekas merged commit f417ab2 into symfony:master Feb 16, 2022

greg0ire deleted the recommend-better-mysql-charset branch February 16, 2022 10:07

greg0ire mentioned this pull request Feb 16, 2022

Remap collate to collation when appropriate doctrine/DoctrineBundle#1471

Merged

greg0ire mentioned this pull request Mar 2, 2022

False collation on current 2.6.x-dev 89b7ed6 version doctrine/DoctrineBundle#1480

Closed

alexander-schranz mentioned this pull request Mar 3, 2022

Fix utf8mb4 charset for mysql database and update recipes sulu/skeleton#160

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Specify charset when using MySQL #1059

Specify charset when using MySQL #1059

greg0ire commented Feb 16, 2022 •

edited

Loading

github-actions bot commented Feb 16, 2022 •

edited

Loading

nicolas-grekas commented Feb 16, 2022

Specify charset when using MySQL #1059

Specify charset when using MySQL #1059

Conversation

greg0ire commented Feb 16, 2022 • edited Loading

github-actions bot commented Feb 16, 2022 • edited Loading

How to test these changes in your application

Diff between recipe versions

doctrine/doctrine-bundle

nicolas-grekas commented Feb 16, 2022

greg0ire commented Feb 16, 2022 •

edited

Loading

github-actions bot commented Feb 16, 2022 •

edited

Loading